Scalable Traffic Signal Controls Using Fog-Cloud Based Multiagent Reinforcement Learning

Optimizing traffic signal control (TSC) at intersections continues to pose a challenging problem, particularly for large-scale traffic networks. It has been shown in past research that it is feasible to optimize the operations of individual TSC systems or a small collection of such systems. However, it has been computationally difficult to scale these solution approaches to large networks partly due to the curse of dimensionality that is encountered as the number of intersections increases. Fortunately, recent studies have recognized the potential of exploiting advancements in deep and reinforcement learning to address this problem, and some preliminary successes have been achieved in this regard. However, facilitating such intelligent solution approaches may require large amounts of infrastructure investments such as roadside units (RSUs) and drones, to ensure that connectivity is available across all intersections in the large network. This represents an investment that may be burdensome for the road agency. As such, this study builds on recent work to present a scalable TSC model that may reduce the number of enabling infrastructure that is required. This is achieved using graph attention networks (GATs) to serve as the neural network for deep reinforcement learning. GAT helps to maintain the graph topology of the traffic network while disregarding any irrelevant information. A case study is carried out to demonstrate the effectiveness of the proposed model, and the results show much promise. The overall research outcome suggests that by decomposing large networks using fog nodes, the proposed fog-based graphic RL (FG-RL) model can be easily applied to scale into larger traffic networks

Scalable Traffic Signal Controls Using Fog-Cloud Based Multiagent Reinforcement Learning

Optimizing traffic signal control (TSC) at intersections continues to pose a challenging problem, particularly for large-scale traffic networks. It has been shown in past research that it is feasible to optimize the operations of individual TSC systems or a small collection of such systems. However, it has been computationally difficult to scale these solution approaches to large networks partly due to the curse of dimensionality that is encountered as the number of intersections increases. Fortunately, recent studies have recognized the potential of exploiting advancements in deep and reinforcement learning to address this problem, and some preliminary successes have been achieved in this regard. However, facilitating such intelligent solution approaches may require large amounts of infrastructure investments such as roadside units (RSUs) and drones, to ensure that connectivity is available across all intersections in the large network. This represents an investment that may be burdensome for the road agency. As such, this study builds on recent work to present a scalable TSC model that may reduce the number of enabling infrastructure that is required. This is achieved using graph attention networks (GATs) to serve as the neural network for deep reinforcement learning. GAT helps to maintain the graph topology of the traffic network while disregarding any irrelevant information. A case study is carried out to demonstrate the effectiveness of the proposed model, and the results show much promise. The overall research outcome suggests that by decomposing large networks using fog nodes, the proposed fog-based graphic RL (FG-RL) model can be easily applied to scale into larger traffic networks

Large Network Multi-Level Control for CAV and Smart Infrastructure: AI-Based Fog-Cloud Collaboration

69A3551747105The first part of this study addresses the use of fog-cloud architecture for a deep reinforcement learning-based control framework and presents a case study involving urban traffic dynamic rerouting. Past work has shown that dynamic rerouting can mitigate traffic congestion and can be facilitated using emerging technologies such as Deep Reinforcement Learning (DRL) and fog-computing. However, two unaddressed challenges include the immense size of the action space associated with urban road networks, and the impairment of learning efficiency engendered by the large size of THE network information. Therefore, this project proposes a two-stage model that combines GAQ (Graph Attention Network \u2013 Deep Q Learning) and EBkSP (Entropy Based k Shortest Path) overlying a fog-cloud information architecture, for higher learning efficiency by shrinking action space and selecting relatively important information to reroute vehicles in a dynamic urban environment. First, the GAQ analyzes the traffic conditions and EBkSP assigns a route to each vehicle based on two criteria. Using a case study, the proposed model is tested and the results demonstrate the efficacy of the model for rerouting vehicles in a dynamic manner. The second part of the study uses fog-cloud based multiagent reinforcement learning scalable for controlling a specific class urban transport systems \u2013 traffic signal systems. Optimizing traffic signal control (TSC) at intersections continues to pose a challenging problem, particularly for large-scale traffic networks. While it is feasible to optimize the operations of individual TSC systems or a small number of such systems, it is computationally difficult to scale these solution approaches to large networks partly due to the curse of dimensionality that is encountered as the number of intersections increases. Fortunately, recent studies have recognized the potential of machine learning tools address this problem. However, facilitating such intelligent solution approaches may require unduly large investments in infrastructure such as roadside units (RSUs) and drones in order to ensure thorough connectivity across all intersections in large networks, an investment that may be financially burdensome to road agencies. As such, this study builds on recent work to present a scalable TSC model that may reduce the number of required enabling infrastructure in this problem context. This study uses graph attention networks (GATs) to serve as the neural network for deep reinforcement learning, which aids in maintaining the graph topology of the traffic network while disregarding any irrelevant or unnecessary information. A case study is carried out to demonstrate the effectiveness of the proposed model, and the results show much promise. The overall research outcome suggests that by decomposing large networks using fog-nodes, the proposed fogbased graphic RL (FG-RL) model can be easily applied to scale into larger traffic networks